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Abstract. We discuss some properties of conditional expectation operators, and use these facts to prove an 
interesting counterexample regarding sufficient statistics. In particular, we show that there exists sufficient 
random variables X and Y, such that (X, Y) are jointly not sufficient. We follow the work in [l] and [2], 
presenting complete proofs and supplying missing steps. 

1. Introduction 

For now, we fix a probability space (fi, J 7 , P). Let X g Li(£l,T, P) be an integrable random variable. 
Then, if Q is a sub-er-field (which we simply call a subfield) of J 7 , the conditional expectation of X with 
respect to Q is defined to be the random variable E [A"|(J] that satisfies: 

(1) For all B eg,E[E [X\Q] I B ] = E [Xl B ] 

(2) E[X|£] is ^-measurable 

The existence of the conditional expectation is a standard result that can be found in [5]. We say 
that an operator T on L\ is a conditional expectation operator if T — E[-\Q] for some sub-sigma field 
Q C T. Let {T n } be a sequence of conditional expectation operators and write the sequence of iterates as 
S n = T n ■ ■ ■ T^Tx. In this report we will discuss certain properties of the iterates of a sequence of conditional 
expectation operators. 

Before diving into the material, a notational explanation is in order. Unfortunately in the world of 
probability, capital letters are used to denote random variables, but in the world of functional analysis lower 
case letters seem to be popular for elements of function spaces. Although this paper uses ideas from both 
fields, our ultimate goal is statistical, so we will use capital letters. The letters T and Q are reserved for 
operators on a Hilbert space, and X is always an element of the function space in question. 

An important interpretation of the conditional expectation is that it is an orthogonal projection operator 
onto the space of (/-measurable functions within the Hilbert space £2(^5 J~, P). This perspective will be the 
driving force behind most of the ideas here. For completeness we include the following result. 

Theorem 1.1. Let T be the operator on the Hilbert space L 2 {Q,T,¥) defined by T(X) = E [X\Q]. Then T 
has the following properties. 

(1) T is self-adjoint: T* — T 

(2) T is idempotent: T 2 = T 

(3) I — T is the projection onto (£2(0, G, P)) 

(4) T is a contraction: \\T\ \ < 1 

Proof. These results follow from the basic properties of conditional expectations. See [B]. □ 

2. Some Preparatory Lemmas 
We begin by stating and proving a lemma found in [5] . 
Definition 1. A sequence {a n }^° of real numbers is said to be convex if aj + i < ^(a^ + a i+2 ) for all i > 1. 

Lemma 2.1. Let {a„}J° be a convex and bounded sequence of real numbers. Then 

00 

nA 2 a n — a\ — lim a n 



n—>oc 
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Proof. We note that A 2 a„ = a„ + 2 — 2a„+i + a„ is non-negative, by the assumption of convexity. Thus we 
can rearrange the sum as we wish. In particular, we are free to use Fubini's theorem. 
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oo oo 

EE a: 



n—1 n—1 k—1 

oo oo 



2 a r , 



k—1 n—k 



= ^ ^ Aa n+ i - Aa n 

n— 

oo 

= Aa fc + lim Aa n+i 

' * n— >oo 
fe=l 

= ai — lim a n 

n— >oo 

□ 

Lemma 2.2. Let {a n }^ =0 be a sequence of complex numbers such that c 2 = X^i n l a « ~~ a n+i| 2 < oo. 
Define 



2™ 



fc=i 
T/ien 

(1) sup„>! |a„| < 3 (sup„> |6 n |) + |c| 

(2) //limbec 6„ = a 0; i/ien lim n _>.oo a n = a 

Proof. Suppose without loss of generality that ao = 0. If ao ^ 0, then we can simply subtract ao from each 
term and the proof would still work. Define 



fc=2" 



Then we have that 



2"+'-l 



max \aj-ak\ < } \a k - afe+i 

2"<7<fc<2" + 1 ^— ' 

_ fc=2" 

/ 2" +1 -l 

< 2" \ a k-a k +i\ 



k=2 n 
'2™ +1 -l 



< Y k\a k -a k+1 \ 

\ fc=2" 



\fc=2" 

\c„\ 



Let to be the integer that satisfies 2 m < n < 2 n+1 . Then we have that 

I On I = \b m - 26 m+ i + ^ 2~ m (a k - a n ) 

fc=2"» + l 

< |6 m | +2|6 m+ i| + |c m | 
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Taking suprcmums of both sides of the above, we get that 



sup \a n \ < 3 sup \b n \ + \c\ 

n>l \n>0 J 

To get the second result, suppose that lim^oc b n = a = 0. Then we have that 
lim fl„ = lim \b m - 2b m+ i + 2~ m (a k -a n ) 

n— >cc m— >oo \ * — ' 

\ fc=2™ + l 



= lim b m - lim 2b m+1 + lim V] 2 m (a k - a n ) 

fe=2™ + l 

= 

□ 

Lemma 2.3. Let T be a linear self-adjoint operator with norm \ \T\ \ < 1 on a Hilbert space %. Then yX 6 H, 

oo 

J2 \\T n (X) - T n+2 (X)\\ 2 < \\X\\ 2 

n=l 

Furthermore, the L 2 -limit of T 2n exists and is an orthogonal projection operator. 

Proof. Fix some IeM and define a n = \\T n (X)\\ 2 . Since ||T|| < 1, we have that the sequence {a n }f is 
bounded. This sequence is also convex, since (by using self-adjointness of T) 

A 2 a n = a n+2 — 2a n+ i + a n 

= ||T" +2 pO|| 2 - 2||T" +1 (X)|| 2 + ||T"pO|| 2 

= ||T™+ 2 (X)|| 2 - 2(T n+1 (X),T n+1 (X)) + ||T"(X)|| 2 

= ||T" +2 (X)|| 2 - 2(T n {X) 1 T n+2 (X)) + ||T n (X)|| 2 

= (T n (X) - T n+2 (X),T n (X) - T n+2 (X)) 

= \\T n {X)-T n+2 {X)\\ 2 

> 

Thus by applying Lemma 1, we get that 

oo 

Y / n\\T n (X)-T n+2 (X)\\ 2 = \\T(X)\\ 2 - lim ||T"(X)|| 2 

n=l 

However, since ||T|| < 1, we have that ||T(X)|| < \\X\\. By using this, and positivity of the limit, we get 
that 



J2 \\T n (X) - T n+2 (X)\\ 2 < ||X|' 2 

n=l 

^2n\ 



To show convergence of {T 2n }, we show that it is a Cauchy sequence. 

lim \\T 2n (X) -T 2m (X)\\ 2 = lim (T 2n (X) - T 2m (X), T 2n (X) - T 2rn (X)) 

n.rn— >oo n.m— >oo 

lim \\T 2n {X)\\ 2 -2{T 2n {X),T 2m {X)) + \\T 2m {X)\f 

n.m— >oo 

But we also have that by using the self-adjointness of T we get 

a n+m = ||T"+ m pO|| 2 

= (T n+m (X),T n+m (X)) 

= (T m (X),T 2n+m (X)} 

= (T 2m (X),T 2n (X)) 
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So we have that 

lim \\T 2n (X)~T 2m (X)\\ 2 = lim (a 2n - 2a n+m + a 2m ) 



— lim («2n — Q"n+m) + (a>2m — Q-n+m) 
n,m— too 

= o 

By the definition of completeness in a Hilbert space, we have that the sequence {T 2 ™} converges in the 
L 2 -norm to some Q E L 2 . Since T is self-adjoint, so must Q. To obtain idempotence, note that since {7 14 ™} 
is a subsequence of {T 2n }, it must also converge to Q. So for all X E H, we have that 

\\Q 2 (X) - Q(X)\\ < \\Q 2 {X) - T An {X)\\ + \\T An {X) - T 2n (X)\\ + \\T 2n (X) - Q(X)\\ 

Since each of the quantities on the right side converge to 0, Q is idempotent and is hence an orthogonal 
projection. □ 

Theorem 2.4. Let T be a linear self-adjoint operator on L 2 = L 2 (Q,J r , P) with norm \ \T\\ < 1. Suppose 
that X G L 2 and that the following limit exists almost surely 

2" 

(1) lim V 2- n T 2k (X) 

n. — ^rxn ' ^ 



fc=l 



T 2n {X) = Q(X). 

Proof. To prove this, we will use Lemma 3.2. Define a n (X) = T 2n (X). Then to invoke Lemma 3.2 we must 
show that c 2 = 5Z?Li n\a n — a n+ i| 2 < 00 a.s. However, we have that 

r. OO OO r. 

/ Y, n \T 2n (X) -T 2(n+1 \X)\ 2 dF = Y"n / \T 2n {X) - T 2(n+1 \X)\ 2 df 

J** n=l n=l J n 

OO 

= ^Tn\\T 2n {X) -T 2( " +1) (X)|| 2 

n=l 

00 

= -J2 2n \\T 2n (X) - T 2( - n+1 \X)\\ 2 

n=l 

< -\\x\\ 2 
- 2" 11 

< 00 

Above we used the posititivity of the integrand to swap the sum and the integral, and we applied Lemma 
3.3 to the operator T 2 . Thus J2™=i n\T 2n (X) - T 2{ - n+1 \X)\ 2 is finite almost surely. Thus by Lemma 3.2 
we get the desired conclusion. □ 

Theorem 2.5. Let T be a linear self-adjoint operator on L 2 = L 2 (VL, J 7 , P) that satisfies the following two 
conditions 

(1) HTpOH^ILYlh 

(2) ||T(J0|U<|LY|U 

Then for all x £ L 2 we have that lim„„ 5 . 00 T 2n (X) — Q{X). 

Proof. To prove this, we invoke the Riesz-Thorin theorem which can be found in [4] on the interpolation 
LiCiLoo C L 2 . In particular we get that T is a bounded operator on L 2 and that \T\ \ 2 < max (||T||i, | |T| |oo) = 
1. Furthermore, we get the appropriate condition ([lj by chapter 3 of [3]. Thus we conclude by Theroem 3.4 
that lim^ooT 2 ™^) = Q{X). ' ' □ 

3. A Result about Conditional Expectation Operators 

Now that we have the basic lemmas out of the way, we can prove a result about conditional expectation 
operators. We will use this in our discussion about sufficiency in the next section. We note that the authors 
in [2] and p] define a conditional expectation operator E [■ \Q\ to have range Q (the completion of the a- field 
'->')• 
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Theorem 3.1. Let T\ and T2 be conditional expectation operators on (f2, J 7 , P). In other words, let T\ = 
E [• and T2 = E [• \G2\1 where Gi and G2 are subfields of T. Let T^h-x = T\ and T2k = T 2 extend the 
sequence for all positive integers. Then, we have that for all X G L 2 , 

lim S n {X)=E[X\GiDG 2 ] 

n— >oo 

both in Li and almost surely. 

Before proving this theorem, we state and prove a quick lemma. 
Lemma 3.2. Let X be square-integrable. Then 

lim (S n (X) - S n+1 {X)) =0 

both in L 2 and almost surely. Furthermore, sup n>1 \S n (X) — S n +i(X)\ G L2 and has norm less than or equal 
to \\X\\. 

Proof. Suppose that X G L 2 (Q, T, P). Then, we have that 

\\S n (X) - S n+1 (X)\\ 2 = (S n (X)-S n+1 (X),S n (X)-S n+1 (X)) 

= (S n (X), S n (X)) - 2(S n+1 (X), S n (X)) + (S n+1 (X),S n+1 (X)) 

= \\S n (X)\\ 2 - 2(S n+1 (X),S n (X)) + \\S n+1 (X)\\ 2 

= \\S n (X)\\ 2 - 2(T n+1 ■ ■■T l (X),T n • • -TipO) + \\S n+l (X)\\ 2 

= \\S n {X)\\ 2 - 2(T n+1 T n+1 ■ ■■T 1 {X),T n • ■ -TipO) + ||S„+i(X)|| 2 

= \\S n (X)\\ 2 - 2(T n+1 ■ ■■T 1 (X),T n+1 T n ■■■T l (X)) + \\S n+1 (X)\\ 2 

= \\S n {X)\\ 2 -\\S n+1 {X)\\ 2 

Where we used the fact that T n+ \ is self-adjoint and idempotent since it is a projection operator. Then we 
will show that 



J2 \S n (X(u)) - S n+1 (X(u))\ 2 < 00 



n=l 



almost surely. Thus for almost every w£fl, we have that lim^oo (S n (X(uj)) — S n+ i(X(ui))) = 0. To see 
this we use the monotone convergence theorem in the following computation 

N 



E 



J2\Sn(X)-S n+1 (X)f 



n=l 



E lim V|S„(X)-S„ +1 pO| s 

iV— »oo z — ' 

n=l 

N 

J2\Sn(X)-S n+1 (X)f 
n=\ 

J2^[\Sn(X)-S n+1 (X)\ 2 ] 



lim E 



= ]T||s„po-s„ + i(x)|| s 

n=l 

= \\Si(x)\\ 2 

= \\T(X)\\ 2 
< \\X\\ 2 <oo 



In particular we also get that E [|5„(X) — S n+ i(X)\ 2 ~\ — > 0. For the last result, we note that 

1 

/ 00 \ 2 

sup \S n (X) - S n+1 (X)\ < \ S n(X(u)) - S n+1 (X(cj))\ 2 



\n=l 



from which it follows that sup n>1 \S n (X) — S n+ i(X)\ G L 2 and has norm less than or equal to \\X\\. 



□ 
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Proof of Theorem 4-1- Let us define T = TiX^Ti. Then T satisfies the conditions of Theorem 3.5. We also 
have that T 2 ™ = Si n+ \. So we have that limbec S± n+ \(X) = Q(X) almost surely and that sup n>1 |S , 4„ + i(X)| 6 
L 2 . Combining this with the previous lemma, we get that lim„^oo S n (X) = Q(X) both in L 2 and almost 
surely. 

To see that Q = E [• \Gi n G2] we note that by Lemma 3.3, Q is an orthogonal projection. Furthermore, by 
definition of Ti and T 2 , its range is L 2 (fl, Gi n02, P)- However, E [• \Gi ("1 G2] is also an orthogonal projection 
onto the same subspace. Thus Q is the same operator as E [• \Gi n G2] on L 2 {Sl, T, P). □ 



4. Application to Sufficiency 

Once again, we fix our measurable space (fi, J 7 ), and we suppose that we have a collection of probability 
measures {P 7 } for 7 e F. We let N denote the smallest cr-ficld containing the sets N that satisfy P 7 (iV) = 
for all 7 6 T. Let us also denote E 7 to be expectation with respect to the probability measure {P 7 }. 

Definition 2. A a -field G C T is called a sufficient subfield if for each bounded T -measurable function f , 
there is a G-measurable function g that satisfies 

E 7 [/I G ] = E 7 [gI G ] 

for all 7 e T and for all G £ G- In other words, G C J- is a sufficient subfield if there exists a g such that 
g = E 7 [f\G] for all 7 G F. 

We consider the following question. If Q\,Gi,... are sufficient sub-fields of T, which of the following are 
also sufficient: Gi H G2, o-(Gi U G2), C^^LiGn, °~{^^=iGn)- We will show that a(Gi H G2) is not sufficient in 
general. 

Example 1. There is an example where a{G\ U G2) is not sufficient, even though Gi and G2 are. 

Proof. Define = {x = (xi,X2) G M 2 : \x\\ — |x 2 > 0}. Let be the probability measure that assigns 
mass \ to each of the points (x\,x-i), (— x\,x<i), {x\,— X2), (—Xi,—x 2 )- We will work with the family of 
probability measures {Prizes!- 

In this case, we have that Af = {0, £1} since there are no non-trival P^-null sets. This means that in 
particular M is a subfield of any subfield of J ' . 

Let us define the reflection operators R\{X) = (xi, — x 2 ) and i? 2 (X) = (— xi,x 2 ), and then define the sets 
Ai X = {x, Ri(X)} for i — 1,2. Define Gi to be the smallest cr-ficld containing {Ai X : x £ £1} for each i = 1,2. 
Note that the set G is in Gi if and only if there is a countable C C SI such that either G = li xe c{Ai x } 
or G = {yJxec{A\ x }) c . Similarly G is in G2 if and only if there is a countable C C SI such that either 
G = U xeC {^2x} or G = (U xe c{-42x}) C . 

Let us define G = o-(Gi U G2), and also define T to be the smallest cr-ficld containing G and the set 
D = {x : x e Ci,xi = xi}. We can also see that 

T = {(Gx r\D)u (G 2 n D) : G U G 2 e G} 

Thus we have fully specified the probability space we are working with along with a family of probability 
distributions as well as the subfields in consideration. 

We first claim that G = o-(Gi U G2) is not sufficient. Suppose it were suffcient. Then there would exist 
a ^-measurable function g satisfying P X (D D G) = E x [glc] for all G € G and all ie!l. If we consider the 
special case of this statement when G — {x}, then we get that V X (D fl {x}) = E^ [gljn;}]- This statement 
tells us that g is the characteristic function of the set D. However, this is impossible since D is an element 
of the subfield G- The reason why D cannot be in G is that all elements of G are either countable or have 
countable complements as argued earlier. However, both D and D c are uncountable by construction. So we 
get a contradiction, and conclude that the subfield G = o~(Gi UG2) is not sufficient. 

Our second claim is that both Gi and G2 are sufficient subfields of T . We will actually just show that Gi 
is sufficient, and then conclude by symmetry that G2 must also be sufficient. Let G\ and G 2 be arbitrary 
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sets in Q. Define the functions 

fl = IGiOD 

h = lG 2 nD 2 

/ = /l+/2 

gi = h + h(Ri(-)) 
92 = / 2 + / 2 (i2i(-)) 

3 = 2 ^ + ^ 

We claim that both gi and 32 are 5i-measurable. We know that G\ is either countable or cocountable. In 
the case that G\ is countable, then the set {x : g\{X) ^ 0} is countable, and in the case that it is cocountable 
then the set {x : gi(X) ^ 1} is countable. If we consider an arbitrary Borel set B s Br, then we have four 
possibilities: either none of and 1 are in B, both of them are in B, is in B but 1 is not, or 1 is in B but 
is not. In the first case, the preimage of B under gi is the empty set, and in the second case the preimagc 
is Q which are both trivially contained in Gi- In the third case when £ B and I B, then by the previous 
discussion on the countability and cocountability of the sets {x : gi(X) ^ 0} and {x : gi{X) ^ 1}, we see 
that the preimage of B under gi must be contained in Gi- By the exact same argument, we see that the 
fourth case also follows. By the same argument given, we can see that gi is also ^-measurable, therefore g 
is also Gi -measurable. 

Let H be a set in Gi, and let be any probability measure in our family of measures. Then, by 
construction of Gi, we have that 

E* [f(Ri(-))l H ]=E x [f(-)l H ] 

Thus, we have that 



E x [gI H ] = E a 



E, 



\ {9\+92)Ih 



\(J + f(Rl))lH 



= E x [fI H ] 

Thus we can conclude that Q\ is a sufficient subficld. By symmetry of the definitions, G2 is also a sufficient 
subficld. 

□ 

Despite the negative result for unions of sufficient subfields, we use the results about iterates of conditional 
expectation operators to show that the result for intersections is positive under the very mild condition that 
at least one of Gi or G2 contain the {P 7 }-null sets. Note that in the above counterexample, this condition 
was satisfied since we showed that M C Gi and M C £2- 

Theorem 4.1. Suppose that N is contained in one of the subfields Gi or G2 (or both). Then the subfield 
G = Gi H G2 is sufficient. 

Proof. Once again, we work in the general setting of the probability space (SI, J 7 , {P 7 }) and we work with 
arbitrary sufficient subfields of J 7 , Gi and G2 ■ We can suppose without loss of generality that N C G2 ■ If not 
we just swap Gi and G2- Let us define the sequence of er-fields as follows. For k > 1 define 

!?2fc-i = Gi 
G2k = G2 

Fix a function / that is -F-measurable and bounded. By the definition of sufficiency, there exists a function 
gi satisfying 

gi = E 7 [f\Goo] 

for all 7 e T. Similarly, by use of sufficiency again, we can inductively define g n to be the function that 
satisfies 

g n = E 7 [,gn-l|<?oo] 
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for all 7 G T. Let us define the function g(X) and h(X) pointwise as follows 

(v\ = { lim ™^°° 92n-i(X) if the limit exists 
g ^ ' 1 otherwise 



. . _ J lim„_ ! . 00 g 2 n(^) if the limit exists 
^ ' otherwise 

Thus by applying Theorem 4.1, we see that 

lim g n =E [f\Gi n £2] 

where we denote ft to be the completion of the cr-field ft with respect to the set of probability measures 
{P 7 }. Thus we have that g = E [/|ft n ft] and that ^ £ A/". The reason why this is true 

is that the convergence in Theorem 4.1 is almost surely. In particular we have that g — h is ft-mcasurablc 
since ft contains W. Thus by measurability of g — h and h with respect to ft, we can conclude that g is 
also ft-measurable. By construction, g is also ft -measurable. Thus g is ft n ft-measurable. 

We then note that ft n ft D ft H ft and by the tower and idempotence properties of conditional 
expectations, 

e 7 [/| ft n ft] = e 7 [E 7 [/| ft n ft] I ft n ft] 
= E 7 [. 9 |ftnft] 

= 9 

Thus we see that the subficld ft n ft satisfies the definition of sufficiency □ 



We now consider the remaining two questions concerning the sufficiency of r\^ =1 Q n and cr(U^L 1 ft l ). By 
the counter example given, we conclude that the second subfield may not be sufficient in general, by taking 
fti = ft for n > 2 and then invoking the above example. However, we will answer the question about the 
sufficiency of flj^ftj affirmatively. 

Theorem 4.2. Let {G n }n°=i be a countable collection of sufficient subfields. Then if Q n D Gn+1 f or a ^ 
n> 1, then n^ =1 Q n is sufficient. 

Proof. By definition of sufficiency, if we are given any bounded .F- measurable function /, there exists a 
fti-measurable function g n such that g n = E 7 [f\Q n ] for all 7 6 V. Define the function g pointwise by 

, , _ J limn^oo g n (X) if the limit exists 
^ ' [0 otherwise 

Suppose that we are in the first situation, where we have a decreasing sequence of cr-fields. Then we have 
that g is fl^jftj-measurable. By the continuity of conditional expectations, we have that 

E 7 [/| n~ ! Gn] = lim g n 

n— >oo 

Thus g = E 7 [f\ n^° =1 G n ] for all 7 e T. So we conclude that D^Gn is a sufficient subfield. □ 



We combine the above with the theorem of the sufficiency of the intersection of two subfields to get a 
proof for the sufficiency of a countable intersection below. 

Theorem 4.3. Let {G n }n°=i be a countable collection of sufficient subfields such that M d Gn for all n > 1. 
Then ri^ =1 G n is also sufficient. 

Proof. Define TL n = (~l^ =1 ft. Then, by applying induction to Theorem 5.1, we see that for each n > 1, 
H n is sufficient. Since V. n D H n +i, we conclude by Theorem 5.2 that f~l£L 1 'H n is also sufficient. But 
n^Lifti = n^° =1 ^„, so D^Gn is a sufficient subfield. □ 
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